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Response to Amendment 

1. Applicants' response to the last Office Action, filed 07/07/2007 has been entered 
and made of record. 

Claim Objections 

2. Claims 1 ,4,5, 1 3-1 8 objected to because of the following informalities: 

In Claim 1, ''substituting data.," is unclear because ''generating a tiole-image,." 
does the same thing. Examiner suggests deleting "substituting data,,". 
Appropriate correction is required. 

In Claim 13, change "data to to zero" to - data to zero 

Ciaim Rejections - 35 USC § 112 

3. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and 
process of making and using it, in such full, clear, concisi, and exact terms as to enable any 
person skilled in the art to which it pertains, or with which it is most nearly connected, to make 
and use the same and shall set forth the best mode contemplated by the inventor of carrying out 
his invention. 

4. Claims 1,4,5,17,18 rejected under 35 U.S.C. 1 12, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. 
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0 Regarding Claims 1,17: 

setting sub-sampled pixel values of zero to the average value of the non-zero 
sub-sampled pixel values. 

It's unclear how zeros got into the sub-sampled hole image, since zeros were 
completely ignored during averaging to fomn the sub-sampled hole image. 

5. Claims 11,16-18 rejected under 35 U.S.C. 1 12, first paragraph, as failing to 
comply with the enablement requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

0 Regarding Claim 11: 

dividing the sum of pixel values by a number of non-zero pixels, to obtain the 

i 

sub-sampled pixel value. 

What if a number of non-zero pixels = 0? Division by zero is undefined. 
0 Regarding Claim 16: 

an average of the non-zero pixels in contiguous, non-overlapping pixel 
neighborhoods. 

If the number of non-zero pixels in a pixel neighborhood=0, it triggers division by zero. 
Applicant should look at Fig9 754->755 case. 



6. The following is a quotation of the second paragraph of 35 U.S.C. 1 1 2: 
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The specification sliall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

7. Claim 5 recites the limitation "non-identified". There is insufficient antecedent 
basis for this limitation in the claim. In Claim 1, all pixels were identified. 

Claim Rejections - 35 USC § 103 

8, The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the invention 
was made. 

1 . Claims 1 ,4,5 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ricardo L. de Queiroz, "COMPRESSION OF COMPOUND DOCUMENTS" in view of 
Bloom PGPUB-DOCUMENT-NUMBER: 20040201726, "Digital camera and method for 
balancing color in a digital image". 
0 Regarding Claim 1: 

A method for processing an image comprising: 
Identifying (preprocessor) pixels in the image which are less critical; 
(unused pixels) 

substituting data into identified pixels, (replaced by any color) 
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the data being chosen to provide a desired characteristic (compression) for 
processing the image; 

(Page 210, column 2, "Since each layer (FG or BG) may contain unused pixels (since 
the pixels in that position will be selected from the other layer), those can be replaced 
by any color in order to enhance compression. This is the function of the preprocessor. 
The overall diagram is illustrated in Fig. 3.") 

generating a hole-image (P21 1 top U/X image) by setting to zero (X) pixel values 
of pixels identified to be less critical to the image; (unused) 

(Page 210, column 2, "Since each layer (FG or BG) may contain unused pixels (since 
the pixels in that position will be selected from the other layer), those can be replaced 
by any color in order to enhance compression.") 

sub-sampling the hole-image, by averaging non-zero pixel values in pixel 
neighborhoods to obtain sub-sampled pixel values for the sub-sampled hole-image; 
(Queiroz+ Bloom) 

averaging the non-zero (U) sub-sampled pixel values of the sub-sampled hole- 
image to obtain an average (average) value; and setting sub-sampled pixel values of 
zero (X) to the average value of the non-zero sub-sampled pixel values. 
(Page 21 1, "...pixels marked "X" who have at least one U-marked horizontal or vertical 
neighbour is replaced by the average of those neighbours and marked "U" for the next 
pass.") 
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Queiroz discloses everything as described above except, sub-sampling the hole- 
image, by averaging non-zero pixel values in pixel neighborhoods to obtain sub- 
sampled pixel values for the sub-sampled hole-image. 

Bloom discloses, [0043] "...sensing 210 and displaying 230 a low-resolution 
image 1 including, for example, sensing the image 1 using fewer than all of the available 
photosites on the sensor 17, processing fewer than all of the pixels sensed by the 
sensor 17, ... averaging pixel values,., or othen/vise sub-sampling the image 1 If 
some photosites did not receive a signal, those values are zeroes. Then Bloom 
processes not zeroes and averages to subsample. 

As Bloom discloses, in the case where some 2D matrix data is missing — 
photosites with no signal — it is desirable to ignore for image subsampling. Subsampling 
can be done for display, or to get rid of no-signal photosites. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time of the invention, to use 
Bloom*s method in Queiroz, average non-zeros for subsampling, to properly display 2D 
data or to avoid black no-signal holes in an image. 

0 Regarding Claim 4: 

The method of claim 1, wherein the desired characteristic is at least one of a 
compression characteristic and a processing speed. 

(Page 210, column 2, "Since each layer (FG or BG) may contain unused pixels (since 
the pixels In that position will be selected from the other layer), those can be replaced 
by any color in order to enhance compression. This is the function of the preprocessor." 
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Page 211, column 1 , "Given the preprocessor just described, our goal is to find the best 
mapping (input block to Mask block) which will optimize compression in a rate-distortion 
(RD) sense. Rate is given in bits necessary to encode all 3 layers.") 

0 Regarding Claim 5: 

The method of claim 1, wherein the substituted data is an average of data values 
of the non-identified pixels. 

(Page 21 1, column 1 , "If there is a mix of U- and X-marked pixels, we follow a multi- 
pass algorithm: in each pass, pixels marked "X" who have at least one U-marked 
horizontal or vertical neighbour is replaced by the average of those neighbours and 
marked "U" for the next pass.") 



2. Claims 6-9 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Queiroz, "COMPRESSION OF COMPOUND DOCUMENTS" in view of 

Gonzalez, "Digital Image Processing, 2/E" {Prentice Hall. 2002. ISBN 0201 180758} and 
further in view of 

Mukherjee PGPUB-DOCUMENT-NUMBER: 20030133617, "Coder matched 
layer separation and interpolation for compression of compound documents". 
0 Regarding Claim 6: 

A method for processing an image to fonri a background plane and N-binary 
foreground planes, comprising: 

Inserting (replace) zeroes (X-marked pixels can be replaced by anything) 
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into pixel data for pixels in the background plane (BG) corresponding to areas (unused 
pixels) which have been placed into one of the N-binary foreground planes, (Fig 1 , 3*^^ column. 
Fig 2: FG) to generate a hole-image; 

[Page 210, column 2, "Since each layer (FG or BG) may contain unused pixels (since 
the pixels in that position will be selected from the other layer), those can be replaced 
by any color in order to enhance compression." "any color" includes black color, which 
is zero valued. Page 210-21 1 , "By inspecting the binary mask, it labels the input block 
pixels as useful (U) or "don't care" (X). The X-marked pixels can be replaced by 
anything else since they are not going to be used for decompression."] 

sub-sampling (1 : 1 subsampling included) the hole-image to obtain one or more blocks 
of sub-sampled pixel values, 

(P210 col2 bottom, "each 8x8 input pixel block". P211 Col1 top: "block") 

each of the sub-sampled pixel values having a non-zero value if a corresponding 
neighborhood has at least one non-zero pixel value, 

(This is a conditional do-nothing step. Queiroz does not change non-zeros.) 

or a zero value if the corresponding neighborhood has all zero pixel values] 
Queiroz discloses everything as described above except, each of the sub- 
sampled pixel values having a zero value if the corresponding neighborhood has all 
zero pixel values. This is a min filter in image processing. The min filter replaces the 
center pixel with the minimum of its neighborhood. 
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Gonzalez discloses a min filter (P235, 5.3-9); he says it reduces salt noise. A 
non-zero center pixel surrounded by zero pixels nnaybe considered salt noise — it's a 
dot. As Gonzalez discloses, it is desirable to reduce salt noise using a min filter. De- 
noising improves image quality. Therefore, it would have been obvious to one of 
ordinary skill In the art at the time of the invention, to use Gonzalez' method in Queiroz, 
remove salt noise using min filter; this improves image quality and compression — 
compression because a single color is compressed better than different colors (Page 
210 column 2, "those can be replaced by any color in order to enhance compression." 
Page 1 1 col1 top, "replace the unused parts of a block with data that will produce a 
smooth block.") 



Queiroz discloses only some of, averaging color values (average of those 
neighbours^ of non-zero (U) sub-sampled pixel values in each of the blocks to obtain a block 
average color value for each of the blocks; (Queiroz averages across neighborhoods, not 
blocks) and substituting (replaced) sub-sampled pixel values (1:1 sub-sampling) of each of 
the blocks that are equal to zero (X) to the block average color value of each of the blocks. 
P21 1 top, "X-marked pixels can be replaced by anything ...pixels marked "X" who have 
at least one U-marked horizontal or vertical neighbour is replaced by the average of 
those neighbours and marked "U". Queiroz does not disclose averaging across a block. 

Mukherjee PGPUB-DOCUMENT-NUMBER: 20030133617 discloses, [0069] 
[0108] "[0070]... assigns the irrelevant pixels the average value of the relevant pixels in 
the selected block. This interpolation procedure tends to be considerably faster 
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As Mukherjee discloses, assigning irrelevant pixels the average value of a block 
is fast. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to use Mukherjee method in Queiroz, assign "X" pixels the average 
value of "U" pixels in its block, when fast speed is needed. 

0 Regarding Claim 7: 

The method of claim 6. further comprising: identifying a previous bloci< (P21 1 
col1 "previous block") based on a predetermined criterion; fJPEG's DC DPCM) 

and substituting an average color value of the previous block (average of the 
previous block) for sub-sampled pixel values in a block in which all of the sub-sampled 
pixel values are zero. 

(Page 21 1, "If there are 64 X-marked pixels, the block is unused and we output a flat- 
block whose pixels have the average of the previous block.) 

0 Regarding Claim 8: 

The method of claim 7, wherein the previous block is the previous block as . 
defined by the JPEG (JPEG) order of blocks within a minimum coded unit. (8x8) 
(Page 210, column 2, "assume layers have same dimensions, and the encoder for FG 
and BG layers is JPEG. For each 8x8 input pixel block the preprocessor receives a 
block of equal dimensions of binary data." 
Page 21^1 , column 1 , "because of JPEG's DC DPCM".) 
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0 Regarding Claim 9: 

The method of claim 6, further comprising one or more of: adjusting the image 
according to predefined requirements; and 

(Page 210, column 2, "Since each layer (FG or BG) may contain unused pixels (since 
the pixels in that position will be selected from the other layer), those can be replaced 
by any color in order to enhance compression." "any color" includes black color, which 
is zero valued. 

Page 210-21 1 , "By inspecting the binary mask, it labels the input block pixels as useful 
(U) or "don't care" (X). The X-marked pixels can be replaced by anything else since they 
are not going to be used for decompression.") 

setting a chroma value of a pixel to a midpoint in its allowed range when a 
luminance value of the pixel is at a maximum of its allowed range. 

3. Claims 10.1 1 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Queiroz, "COMPRESSION OF COMPOUND DOCUMENTS" in view of 
Gonzalez, "Digital Image Processing, 2/E" {Prentice Hall. 2002. ISBN 0201 180758} as 

applied to claim 6 above, and further in view of 

Bloom PGPUB-DOCUMENT-NUMBER: 20040201726, "Digital camera and 

method for balancing color in a digital image". 
0 Regarding Claims 10,11: 
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Quelroz and Gonzalez everything as described above except, sub-sampling the hole- 
image comprises: averaging one or more pixel values within a neighborhood of pixels to 
obtain a sub-sampled pixel value that corresponds to the neighborhood. 

Bloom discloses, [0043] "...sensing 210 and displaying 230 a low-resolution 
image 1 including, for example, sensing the image 1 using fewer than all of the available 
photosites on the sensor 17, processing fewer than all of the pixels sensed by the 
sensor 17, ... averaging pixel values... or othenwise sub-sampling the image 1." If 
some photosites did not receive a signal, those values are zeroes. Then Bloom 
processes not zeroes and averages to subsample. 

As Bloom discloses, in the case where some 2D matrix data is missing — 
photosites with no signal, in our case holes or X values— it is desirable to ignore them 
for image subsampling. Subsampling can be done for display, or to get rid of no-signal 
photosites. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention, to use Bloom's method in Queiroz, average non-zeros for 
subsampling, to properly display 2D data or to avoid black no-signal holes or X's in an 
image. 

4. Claim 12 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Queiroz, "COMPRESSION OF COMPOUND DOCUMENTS" in view of 
Gonzalez, "Digital Image Processing. 2/E" {Prentice Hall. 2002. ISBN 

0201 180758}, and further in view of 
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Bloom PGPUB-DOCUMENT-NUMBER: 20040201726, "Digital camera and 
method for balancing color in a digital image" as applied to claim 10 above, and further 
in view of 

Richardson, "H.264 and MPEG-4 Video Compression: Video Coding for Next- 
generation Multimedia", p16-17. 
0 Regarding Claim 12: 

The method of claim 10, wherein the neighborhood of pixels is a 2x2 
neighborhood for luminance data, 
(Queiroz. P21 1 Col1 bottom: "2x2".) 

Queiroz discloses everything as described above except a 4x4 neighborhood for 
chroma data. 

Richardson discloses pi 6, "Cr and Cb component may be represented with a 
lower resolution than Y because HVS is tess sensitive to color than luminance." P17, 
"4:2:2 sampling" in Y:Cb:Cr. So, in relation to Y, Cr is further subsampled by a factor of 
2. Therefore, if Y uses 2x2 neighborhoods for averaging in Queiroz, Cr needs to use 
4x4. 

For compression it's desirable to further subsample Cr in relation to Y, since HVS 
is less sensitive to Cr. Therefore, it would have been obvious to one of ordinary skill in 
the art at the time of the invention to use Richardson's method in the combined method 
of Queiroz+Gonzalez+Bloom to further subsample Cr to enhance compression by 
taking advantage of HVS (human visual system). 
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5. Claims 13,16-21 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Queiroz, "COMPRESSION OF COMPOUND DOCUMENTS" in view of 
Mukherjee PGPUB-DOCUMENT-NUMBER: 20030133617, "Coder matched 

layer separation and interpolation for compression of compound documents". 
0 Regarding Claim 13: 

An apparatus that processes an image, comprising: 

a memory that stores image data, 
(Queiroz. Page 210, column 2, "For the storage, archiving and general interchange of 
MRC-encoded image data, the TIFF-FX file format has been proposed.") 

and selector data (P210 bottom, "binary mask") 

wherein the selector data identifies (preprocessor) less critical portions of the 
image data; 
(unused pixels) 

a processor (preprocessor) that sets less critical portions (unused pixels) of the 
image data to zero (X) based on the selector data; (binary mask) 
(Page 210, column 2, "Since each layer (FG or BG) may contain unused pixels (since 
the pixels in that position will be selected from the other layer), those can be replaced 
by any color in order to enhance compression. This is the function of the preprocessor. 
The overall diagram is illustrated in Fig. 3." "any color" includes black color, which is 
zero valued. 



Application/Control Number: 10/776,603 Page 15 

Art Unit: 2624 

Page 210 bottom, "By inspecting the binary mask, it labels the input block pixels as 
useful (U) or "don't care" (X). The X-marked pixels can be replaced by anything else 
since they are not going to be used for decompression.") 

a sub-sampling processor that sub-samples hole-image data and 
(1:1 subsampling. Please specify that subsampled image has less pixels than the 
original.) 

averages the non-zero data values (P21 1 coll "average of those neighbours") 

in a block of the sub-sampled hole-image data to obtain a block average value; 
(Queiroz not discloses^ 

and a pixel substitutor (preprocessor) which substitutes (replaced) the block 
average value CQueiroz not discloses^ of the non-zero data values (U) for the zero 
values (X) in the sub-sampled (1:1 subsampling) hole-image data. 

Queiroz discloses everything as described above except averages the non-zero 
data values in a block and substituting that average for non-zero pixels. 

Mukherjee PGPUB-DOCUMENT-NUMBER: 20030133617 discloses, [0069] 
[0108] "[0070]... assigns the irrelevant pixels the average value of the relevant pixels in 
the selected block. This interpolation procedure tends to be considerably faster ...". 

As Mukherjee discloses, assigning irrelevant pixels the average value of a block 
is fast. Therefore, it would have been obvious to one of ordinary skill in the art at the 
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time of the invention to use Mukherjee method in Queiroz, assign "X" pixels the average 
value of "U" pixels in its block, when fast speed is needed. 

6. Claim 16 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Queiroz, "COMPRESSION OF COMPOUND DOCUMENTS" in view of 

Mukherjee, PGPUB-DOCUMENT-NUMBER: 20030133617, "Codermatched layer 
separation and interpolation for compression of compound documents", and further in view of 

Bloom, PGPUB-DOCUMENT-NUMBER: 20040201726, "Digital camera and 
method for balancing color in a digital image". 

0 Regarding Claim 16: 

Queiroz discloses everything as described above except, The apparatus of claim 
15, wherein the sub-sampling processor sub-samples the hole-image data by setting 
the sub-sample pixels to values equal to an average of the non-zero pixels in 
contiguous, non-overlapping pixel neighborhoods. 

Bloom discloses, [0043] "...sensing 210 and displaying 230 a low-resolution 
image 1 including, for example, sensing the image 1 using fewer than all of the available 
photosites on the sensor 17, processing fewer than all of the pixels sensed by the 
sensor 17, ... averaging pixel values... orothenwise sub-sampling the image 1." If 
some photosites did not receive a signal, those values are zeroes — holes. Then Bloom 
processes not zeroes and averages to subsample. 

As Bloom discloses, in the case where some 2D matrix data is missing — 
photosites with no signal are in our case holes or X values — it is desirable to ignore 
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them for image subsampling. Subsampling can be done for display, or to get rid of no- 
signal photosites. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time of the invention, to use Bloom's method in Queiroz, average non-zeros 
for subsampling, to properly display 2D data or to avoid black no-signal holes or X's in 
an image. 

0 Regarding Claim 17: 

The apparatus of claim 16, further comprising: a hole-filler (preprocessor) that 
identifies previous blocks (P21 1 col1 "previous block") based on predefined criteria, 
^JPEG's DC DPCM) 

and replaces (replaced) zero values (X) of the blocks of sub-sampled hole-image 
data with a previous block average value, (average of the previous block) when the 
blocks of sub-sampled hole-image data consist entirely of zeroes. 
(Page 211, column 1 , "The block-wise pre-processor we used in this paper works as 
follows. If there are 64 X-marked pixels, the block is unused and we output a flat-block 
whose pixels have the average of the previous block.) 

0 Regarding Claim 18: 

The apparatus of claim 17, wherein the previous block is the previous block as 
defined by the JPEG order of blocks within a minimum coded unit. 

(Page 210, column 2, "assume layers have same dimensions, and the encoder for FG 
and BG layers is JPEG. For ieach 8x8 input pixel block the preprocessor receives a 
block of equal dimensions of binary data." 
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Page 210-211, "By inspecting the binary mask, it labels the input block pixels as useful 
(U) or "don't care" (X). The X-marked pixels can be replaced by anything else since they 
are not going to be used for decompression." "any color" includes black color, which is 
zero valued. 

Page 211, column 1, "because of JPEG's DC DPCM".) 

0 Regarding Claim 19: 

A computer-readable medium having computer-readable program code embodied 
therein, the computer-readable program code performing the method of claim 1. 
(Page 210, column 2, "The compressed layers are then packaged in a format, such as 
TIFF-FX [21] or as an ITUT MRC [19] data stream for delivery to the decoder. At the 
decoder, each plane is retrieved, decompressed, processed (which might include 
scaling) and the image is composed using the MRC imaging model. 
MRC was originally approved for use in Group 3 color fax and is described in ITU-T 
Recommendation T.44. For the storage, archiving and general interchange of MRC- 
encoded image data, the TIFF-FX file format has been proposed.") 

0 Regarding Claim 20: 

A xerographic marking device using the method of claim 1. 
(Page 210, column 2, "MRC was originally approved for use in Group 3 color fax and is 
described in ITU-T Recommendation T.44.") 
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0 Regarding Claim 21: 

A digital photocopier using the method of claim 1 . 

(Page 210, column 2, "MRC was originally approved for use in Group 3 color fax and is 
described in ITU-T Recommendation T.44.") 

Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. MacLeod (US-PAT-NO: 5778092) discloses, "Method and 
apparatus for compressing color or gray scale documents." 

Debargha Mukherjee, Christos Chrysafis, and Amir Said disclose, "JPEG2000- 
Matched MRC compression of compound documents," Proc. IEEE Int. Conf. Image 
Processing, vol. 3, pp. 73-76, Sept. 2002. (PDF). 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Max Shikhman whose telephone number is (571) 270- 
1669. The examiner can normally be reached on Monday-Friday 8:30AM-6:00PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, JINGGE WU can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Max Shikhman 
8.29.20.07 



